Amortized SVGD (Stein Variational Gradient Descent)
Stein variational gradient descent (SVGD) [1] is a deterministic, gradient-based sampling algorithm for approximate inference. Given a probability density function by a simple iterative update of form. Compared with Monte Carlo methods, SVGD can achieve good approximation even with a very small number of particles. A simple way to see this is to note that when using only a single particle ( (a.k.a. maximum a posteriori (MAP)), which is often found to be a useful approximation in many difficult practical problems. SVGD with more particles interpolates between gradient descent and approximate inference and provides better uncertainty assessment.